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Test What You Fly? 

by Don Margolies 

It was the first time on any NASA project I know of that all the instruments on 
an observatory came off for rework or calibration after the full range of environ- 
mental tests, and then were reintegrated at the launch center without the benefit 
of an observatory environmental retest. 

Perhaps you've heard the expression, "Test what you fly, fly what you test"? In 
theory, it's hard to argue with that. In this case, I was willing to take the risk of 
not testing what I flew. As the project manager for the Advanced Composition 
Explorer (ACE) mission, I was the one who ultimately decided what risks to take, 
just as it was my responsibility to get buy-in from the stakeholders. 

Whatever Possessed Me? 

The ACE observatory had a suite of nine instruments and an electronics box that 
interfaced with a number of the instruments. We had planned from the outset of 
the project to remove three of the instruments to have microchannel plates 
replaced and be re-calibrated. Our implementation plans took this into account, 
and we developed retest programs for the individual instruments. Subsequent 
developments changed this plan significantly, when I authorized the removal of 
all instruments and the electronics interface box. 

Several people on the project thought I was crazy. Why do this? We had gone 
through our environmental test programs successfully, and everything seemed to 
be working okay. We had previously stated that the only reasons for removing an 
instrument after environmental testing would be either because it was one of the 
three mentioned earlier, or because something had broken and needed to be fixed. 
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Normally, I would have taken a pragmatic approach: You’re on the observatory, 
your instruments are working, and good enough is good enough. " On the other 
hand, we had more than adequate slack in the schedule, and we were coming in 
$30 million under budget- amazing, I know. The question then was, what could 
we do to make the science better? Given that we had the time, given that we had 
the money, one answer to this question was! better calibration. For those who just 
do a marginal job of calibration prior to testing, the alternative is to calibrate 
again in orbit. Calibration in orbit takes a long time to do, and it's not as precise 
as it is on the ground. So there really was a net benefit to be gained from the sci- 
ence of doing this. If our scientists had the opportunity to tweak and calibrate 
their instruments on the ground, they would likely get better science in space. In 
order to provide a proper return on our $100M NASA investment, ACE— an 
Office of Space Science mission -had to perform on all cylinders, so to speak. 
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Test What You Fly? (cont'd) 


Dealing With My Stakeholders 

How do you evaluate the risk of putting an instrument back on without retest- 
ing it under vibration? That was the question my management put to me. When 
I approached them about this, they thought I was crazy. "Don, you are crazy," 
they told me, in fact--but I had known I was going to hear this, and was prepared 
to explain. 

“ How do you evaluate 
the risk of putting an 
instrument back on 
without retesting it 
under vibration? 55 

The way dial the spacecraft was designed it made the job of removing the instru 
ments and reintegrating them very simple. The instruments were mounted on the 
outside of the spacecraft and easily accessible. You basically disconnected the con 
nectors and there weren't that many removed the mounting bolts, and lifted off 
the boxes. When you re bolted them down, you made sure of their mechanical 
integrity, and you did functional tests on each of the instruments to verify that they 
were working okay. The solar panels were off anyway at this point, so if we had had 
to get into the guts of the spacecraft that would not have posed a problem. 


Now this may all sound well and good on paper, but you don't persuade your 
management to do something it usually does not want to do just by sounding 
logical. When you want to do something this radically different, you have to be 
cool and clear in how you present the issue. The last thing you want to seem is 
impulsive. Upper management wants to hear that things have been checked out 
according to protocol. They sleep better when they know that protocol has been 
followed ("Test what you fly, fly what you test"), and consequently a project 
manager sleeps better when he knows his management isn't tossing and turning 
about a decision he's made. 


"Yes," I admitted, "when we reintegrate everything we will not have the same 
degree of certainty as we did before the testing. Without another vibration test, 
no one would ever be able to say categorically that these instruments will hold up 
under the stress of launch." 


Ultimately, I was able to get management to buy off on the decision, but not with- 
out first making an independent review of our plans. The review board agreed 
with me that based on the design of the spacecraft reintegration would be "less 
complex" than it could be. The fact that all instruments would be environmen- 
tally tested before they were returned to the spacecraft also helped. But there were 
other stakeholders, including those at headquarters, whom I also had to convince. 
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The Folks Who Were Doing The Work 

The person I was most concerned about was the project manager of the organi- 
zation that was building the spacecraft, the Johns Hopkins Applied Physics 
Laboratory (APL). It was APL's responsibility to reintegrate the instruments, and 
I knew the APL project manager, Mary Chiu, had hardly been delighted when I 
told her what I wanted to do. In fact, it was she who reminded me before any- 
body else that you should "Test what you fly, fly what you test." 



This is the logo for the Advanced Composition Explorer (ACE) mission, 
which was launched in August 1997. ACE is studying low-energy parti- 
cles of solar energy and high-energy galactic particles, and continues to 
collect data that scientists use today to understand our universe. 
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Test What You Fly? (cortt'd) 


Mary never yelled or screamed or jumped up and down and said no, we can't do 
this; and I don't believe she ever said, "Don, you are crazy," (at least, not to my 
face like my management did); but she did voice her displeasure in writing, and 
this was not something I took lightly. 


u The scientists were 
told that if they did not 
get their instruments 
back in time, they 
might not fly. ” 


We talked about it exhaustively, especially in terms of what the impact on her 
team would be. Mary was definitely a key part of the planning process because it 
was going to be up to her people to reintegrate the instruments, as well as to do 
all the other things we needed to do as the launch date approached. Now, I 
wouldn't have changed the plans had I thought the APL team felt I was running 
an impossible risk. Getting Mary's buy-in, albeit a reluctant buy-in, was a major 
precondition for going through with this. 


I left it up to Mary to decide when she needed all the instruments on site. The 
way the instruments' schedules were laid out, they were going to arrive in a 
waterfall fashion so there would be adequate time to integrate them. If they all 
showed up at the door the same day, Mary's team would really have to hustle, and 
I didn't want that. The APL team was working very hard as it was. I had to make 
sure they were taken care of, so Mary's schedule dates were what the scientists 
committed to. 


And Then There Were The Scientists 

In order to even consider this risk in the first place, I had to have complete faith 
in the scientists I was working with. More than any of the others, they were the 
group who would determine whether or not we could accept such a risk. How 
did I guarantee their full cooperation? The scientists were told that if they did not 
get their instruments back in time, they might not fly. They understood that, and 
more importantly, they believed it. I know they did because the Principal 
Investigator, Dr. Stone, stood behind me on this. I insisted that each of the co- 
investigators write a letter to Dr. Stone promising that they would return their 
instrument no later than the date given them. I don't know whether or not they 
would have written such a letter to me, but based on what I had seen so far, I 
knew that once they promised Dr. Stone, no matter what condition the instru- 
ment was in--whether it was fully calibrated or not--it would arrive by the day 
they had agreed it would. 

Our Science Team knew that our primary objective was to launch on schedule. 
They also knew that because of some science overlap in the instruments, we 
would be willing to leave an instrument on the ground if we had to. The truth of 
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the matter was that despite what we had said outwardly, our goal was always to 
fly a complete science suite, with all instruments functioning as well as they 
could, and with all the appropriate calibrations. 

And so it all worked out in the end. The orderly return of the instruments did- 
n't happen exactly as we had planned, but due to the skill and dedication of the 

APL team we reintegrated the instruments at the launch site, and launched ACE 

within four days of the date we had specified three and a half years earlier. This Question 
happened because people were willing to work with one another to make it hap- 
pen. The proof, I believe, is that the instruments have been working for almost When do you know it's okay to 
five years now and are providing wonderful scientific results. break a rule? 


Lessons 

• Project principles should be adjusted to suit particular circumstances. However, 
before you break a rule, consider the ramifications and understand the importance 
of getting buy-in from all those who have a stake in making something happen. 

• Lead by influence rather than authority, by seeking cooperation rather than using 
solicitation, and you are much more likely to get buy-in from your team. 






